[Conventional Technology for Remote Editing]
Image data, particularly motion picture data, is widely used. For example, it is broadcast by television, distributed through a network, or stored on a DVD (Digital Versatile Disc) or other storage medium for sale.
The above image data is generated by editing image data that is generated, for instance, by video recording.
Image data editing is either locally performed at an editing station having raw image data (image data that serves as a material) or remotely performed after transmitting raw image data through a network.
For local editing an editing process is performed by connecting a VTR, which stores raw image data, to an editing apparatus which edits the raw image data, with a short cable that does not cause transmission delay.
As a technology for performing the above editing process a “data transmission system” is disclosed by Japanese Patent Laid-Open No. Hei 10-75413.
When the above technology is used, the editing apparatus transmits a time code request command to the VTR, and the VTR returns a time code to the editing apparatus to ensure that an accurate editing process is performed in accordance with the time code.
Further, scenes (time ranges) are arbitrarily selected from one or more pieces of image data through the use of the time code. The image data contained in the selected scenes are then interconnected to create a piece of edited image data.
A point that serves as a scene connection point is called an edit point. The edit point is usually specified by a time code that indicates the time position of the edit point within the entire image data.
On the other hand, when editing is to be remotely performed, the raw image data is usually compressed before being transmitted to the remote editing machine because the communication capacity of a network is generally smaller than the size of raw image data.
Therefore, even when video recording is performed in a high-definition (high image quality) mode, the remote editing machine performs editing after transcoding the image resolution to a standard-definition (standard image quality) level.
FIG. 36 shows a typical system configuration for the scheme described above. A main editing machine 501 retains raw image data.
A terminal 502 performs an encoding operation to reduce the bit rate by converting the resolution (spatial resolution) of raw image data or by lowering the image quality, maps the raw image data to network packets, and transmits the packets to a network 503.
The image data transmitted to the network 503 is received and decoded by another terminal 504, and then input into a remote editing machine 505.
An editor uses the image data to perform editing with the remote editing machine 505 and generates editing information (editing list).
The editing information is transmitted from the remote editing machine 505 to the main editing machine 501. The main editing machine 501 generates edited image data from the raw image data in accordance with the editing information.
The remote editing system described above is also described in Japanese Patent Laid-Open No. Hei 10-75413.
[Conventional Technology for Image Hierarchzation]
Image data, particularly motion picture data, is large in size. Therefore, it is usually compressed by a certain method before being saved or transmitted.
An MPEG (Moving Pictures Experts Group) compression technology is commonly used as an image compression method.
In recent years, technological development activities have been vigorously conducted to provide means for storing an MPEG stream, which is generated by means of MPEG compression, in IP packets conforming to the IP (Internet Protocol) and distribution the IP packets through the Internet.
The MPEG stream transmitted to the Internet is received and reproduced by PCs (Personal Computers), PDAs (Personal Digital Assistants), cellular phones, and various other communication terminals.
For video-on-demand or live image streaming distribution or video conferencing, video telephoning, or other real-time communication, it is necessary to assume that image data is transmitted/received between communication terminals having different capacities.
More specifically, the data transmitted from an information transmission source may be received and reproduced by a cellular phone or other communication terminal having a low-resolution display and a low-speed CPU (Central Processing Unit) or by a desktop PC, a set-top box, a game machine, or other communication terminal having a high-resolution monitor and a high-speed CPU.
A communication system based on hierarchical encoding or a method of hierarchical encoding of image data to be transmitted/received is conceived as a scheme for efficiently performing an image data transmission/reception process and image display process in accordance, for instance, with the processing capacity of each communication terminal in a situation where data is transmitted to various communication terminals having different processing capacities as mentioned above.
The hierarchically encoded image data is packetized in the aspect which can distinguish the encoded data to be processed only at a communication terminal having a high-resolution display from the encoded data to be processed commonly by a communication terminal having a high-resolution display and a communication terminal having a low-resolution display, and the packetized data is transmitted by the transmitting end.
At the receiving end, the encoded data matching its capacity is selected and processed.
An example of a compression/decompression method that permits hierarchical encoding would be a video stream that is provided by MPEG 4 or JPEG (Joint Photographic Experts Group) 2000.
MPEG 4 incorporates a fine granularity scalability technology and has become profiled. With this hierarchical encoding technology, it is possible to provide distribution in a scalable manner and at low to high bit rates.
On the other hand, JPEG 2000, which is based on a wavelet transform, can achieve hierarchical packetization on the basis of resolution or on the basis of image quality by making effective use of a wavelet transform.
Further, JPEG 2000 provides the Motion JPEG 2000 Part 3, which can handle motion pictures as well as still pictures and save hierarchical data in a file format.
In a conventional data distribution system, the transmitting end has to generate data in different formats depending on the capacity of a data reception terminal and prepare different data in accordance with the transmission rate. However, when the hierarchical encoding process described above is performed, the use of single file data makes it possible to simultaneously distribute data to terminals having different capacities.
When, for instance, image data is to be distributed, the UDP (User Datagram Protocol) is often used for communication through the Internet because real-time capabilities are required. Further, the RTP (Real-time Transport Protocol) is used for a UDP layer to use a defined format for each application, that is, each encoding method.
A technology for handling hierarchical data as described above is described in Japanese Patent Laid-Open No. 2003-152544 (hereafter referred to as Patent Document 2), which discloses a “data communication system.”
The above technology relates to the retransmission of packets containing hierarchically encoded image data. This technology is described below in detail.
For example, a DCT (Discrete Cosine Transform) based technology is used for the distribution of hierarchically encoded data.
This technology provides hierarchization with high and low regions differentiated from each other by performing a DCT process on image data and other data to be distributed, generates packets that are classified into high and low regions, and achieves data distribution.
The UDP differs from the TCP (Transmission Control Protocol)/IP in that the former does not retransmit packets even when packet loss occurs. Therefore, when the UDP is used, packet loss may occur due, for instance, to network congestion.
Patent Document 2 mentioned above proposes a scalable distribution technology based on a wavelet transform technology, which can exercise retransmission control over packet loss.